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Executive  Summary 

Pain  is  a  subjective  experience  that  is 
reflected  in  both  behavioral  and  physiologic 
responses.  It  is  inherently  difficult  to 
objectively  assess  and/or  quantify  [1 , 2],  yet 
appropriate  treatment  and  management  of 
pain  is  predicated  on  adequate  assessment. 

Whenever  possible,  the  existence  and 
intensity  of  pain  are  measured  by  the 
patient’s  self-report  [2,  3].  Patients  who 
cannot  reliably  self-report  their  pain,  due  to 
underlying  medical  conditions  or  concurrent 
treatments,  are  at  significantly  higher  risk  for 
inadequately  managed  pain  [1].  For  example 
patients  in  the  intensive  care  unit,  older 
adults  with  dementia,  patients  with  major 
cognitive  or  communicative  impairments, 
infants,  and  patients  under  general 
anesthesia  may  not  be  able  to  reliably  self- 
report  [1 , 4-7].  For  them,  other  strategies 
must  be  used  to  assess  pain.  Consistently, 
multi-dimensional  approaches  to  pain  assessment  have  proven  superior  to  metrics  focused  on  a 
single  variable  [1, 4].  No  single  clinical  indicator  is  sufficient  to  assess  pain.  Therefore,  it  is 
critical  to  combine  behavioral  pain  assessment  with  physiological  monitoring,  and  to  interpret 
behaviors  in  a  patient  specific  context  [2], 


Figure  1:  The  Multimodal  Objective  Pain 
Assessment  Sensor  System  (MoPASS)  consists 
of  two  wireless  wearable  sensors  that  measure 
clinical  correlates  of  pain  and  aggregate  these 
measures  into  a  single  pain  score. 


There  are  several  important  benefits  to  our  versatile  and  multimodal  approach: 

1)  Multimodal  -  machine  learning  algorithms  include  behavioral  and  physiologic  indicators 

2)  Versatility  -  the  forehead  or  finger  sensor  can  be  removed  due  to  patient  specific  issues 

3)  Specificity  -  the  contribution  of  pain  indicators  to  the  pain  severity  score  can  be  adjusted  for 
different  populations  (e.g.,  movement  may  be  irrelevant  for  intentionally  sedated  patients) 

4)  Fieldability  -  low-cost,  lightweight  system  can  easily  be  deployed  in  a  field  hospital  or  clinic 

5)  Continuous  -  wearable  low-power  sensors  enable  long-term  continuous  monitoring 


During  Phase  I  of  this  project,  we  designed  and  prototyped  a  Multimodal  Objective  Pain 
Assessment  Sensor  System  (MoPASS,  Figure  1).  MoPASS  is  a  wearable  and  low  power 
system  for  rapid  pain  assessment  and  continuous  long-term  monitoring.  The  system  will 
implement  a  multimodal  pain  assessment  algorithm  that  will  be  developed  using  machine 
learning  techniques  based  on  the  results  of  clinical  studies  conducted  during  Phase  II  (see 
Phase  II  Work  Plan).  To  monitor  these  parameters,  we  designed  and  prototyped  a  system  of 
two  lightweight,  comfortable,  wireless,  wearable  sensors,  worn  on  the  forehead  and  finger, 
which  relay  real-time  signals  to  the  MoPASS  software 
developed  in  Phase  I. 

The  prototype  Forehead  Sensor  consists  of  a  functional 
unit  which  records  and  transmits  two  lead  EEG  (Figure  2). 

During  Phase  I  we  identified  NeuroSky  (San  Jose,  CA)  as 
our  preferred  vendor  for  EEG  hardware  component 


Figure  2:  The  EEG  component  of  J 
the  Forehead  Sensor  (courtesy  of 
NeuroSkv.  San  Jose.  CA ) 


because  their  chipsets  provide  access  to  algorithmically  optimized  data.  These  pass-band 
filtered  signals  will  be  used  directly  in  our  MoPASS  algorithms.  The  prototype  that  we  developed 
during  Phase  I  is  capable  of  streaming  at  512  Hz  in  real-time  via  Bluetooth  to  a  nearby 
computer  running  MoPASS  software.  Clinical  studies  in  Phase  II  will  examine  these  parameters 
in  response  to  acute  pain. 


The  prototype  Finger  Sensor  records  and  transmits  tri-axial 
acceleration,  temperature,  and  GSR  (Figure  3).  During 
Phase  I,  we  identified  Shimmer  (Dublin,  Ireland)  as  our 
preferred  vendor  for  GSR  because  it  is  a  validated  device, 
it  has  Bluetooth  capabilities,  and  we  have  had  previous 
positive  experiences  with  Shimmer  sensors.  This  firmware 
uses  a  command  response  communication  protocol  to 
successfully  communicate  with  the  MoPASS  software.  The 
firmware  we  developed  allows  the  MoPASS  software  to 
dynamically  control  and  query  sensor  status,  start  and  stop 

data  streaming,  and  configure  the  sampling  rate  and  gain.  Signals  are  acquired  at  100  Hz. 


Figure  3:  The  movement,  GSR, 
and  temperature  component  of  the 
Forehead  Sensor  (courtesy  of 
Shimmer,  Dublin,  Ireland). 


The  design  specifications  for  the  Finger  Sensor  were  to  measure  tri-axial  acceleration,  skin 
conductance,  heart-rate,  and  skin  temperature.  During  development  we  decided  to  include 
blood  oxygen  saturation  (i.e.,  pulse-ox)  because  optics  was  our  preferred  method  of  recording 
heart-rate  and  using  this  sensing  modality  pulse-ox  was  readily  available  without  any  cost  or 
user  comfort  implications.  We  developed  a  prototype  of  the  Finger  Sensor  by  combining  a 
wireless  pulse  oximeter  with  the  previously  discussed  Shimmer  GSR  sensor  (Figure  4).  Signals 
are  acquired  at  100  Hz. 


The  prototype  of  the  Finger  Sensor  that  we  have  developed  extends  to  the  wrist  (Figure  4); 
however,  in  Phase  II  we  will  design  and  manufacture  a  custom  sensor  to  make  all  relevant 
measurements  directly  from  a  single  sensor  on  the  finger. 


Figure  4:  The  prototype  of  the 
Finger  Sensor  that  was  developed 
during  Phase  I.  During  Phase  II  the 
components  on  the  wrist  will  be 
eliminated  and  all  measurements 
will  be  made  by  a  single  customized 
sensor  worn  on  the  finger. 


The  off-the-shelf  hardware  used  in  Phase  I  does  not  have  an  acceptable  form-factor  for  a 
commercialized  device.  In  Phase  II,  we  will  design  and  manufacture  a  single  forehead  sensor 
that  is  capable  of  making  all  measurements  (GSR,  EEG,  and  acceleration). 

The  MoPASS  software  that  we  developed  during  Phase  I  uses  the  Bluecove  library,  which 
implements  the  Java  Bluetooth  standard  JS-82.  Bluecove  was  developed  by  Intel  and  is  now 
widely  used  in  Java  Bluetooth  applications.  It  supports  multiple  Bluetooth  communication 
protocols,  including  RFCOMM,  which  is  used  by  the  MoPASS  devices.  Using  the  MoPASS 
software  we  are  able  to  query  the  status  of  each  MoPASS  sensor,  control  the  sampling  status  in 
real  time,  parse  incoming  data  streams,  plot  time-series  data,  and  export  data  in  Matlab  or 
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ASCII  delimited  text  formats.  We  have  also  implemented  an  extensible  sensor  data  parsing 
component  in  the  MoPASS  software.  This  allows  a  flexible  architecture  for  incorporating  or 
removing  specific  sensors  in  the  future  without  changing  the  software  architecture. 
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Figure  5:  MoPASS  software  for  wirelessly  acquiring  real-time  signals  from  the  Forehead 
and  Finger  Sensors.  Algorithms  developed  based  on  these  trials  in  Phase  II  will  be  used  to 
modulate  the  pain  intensity  score  shown  at  the  bottom  of  the  screen. 

Thus,  the  key  outcomes  of  our  Phase  I  work  were: 

•  System  design  including  pain  metrics  and  appropriate  wearable  sensing  modalities 

•  A  functional  beta-prototype  consisting  of  two  wearable  sensors  and  software 

•  Clinical  trial  design  and  selection  of  a  clinical  site  for  trials  to  be  conducted  in  Phase  II 
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First  Phase  I  Outcome:  Identification  of  Pain  Measures  &  MoPASS  Sensing  Modalities 

Consistently,  multi-dimensional  approaches  to  pain  assessment  have  proven  superior  to  metrics 
focused  on  a  single  variable  [1,  4].  No  single  clinical  indicator  is  sufficient  to  assess  pain. 
Behaviors  do  not  specifically  reflect  pain  intensity;  rather  they  indicate  distress,  which  could  be 
physiologic  or  emotional  in  nature  [8].  Physiological  parameters  can  change  in  response  to 
many  factors  other  than  pain,  and  the  absence  of  changes  in  these  signals  does  not  indicate  the 
absence  of  pain  [2]. 

Several  tools  have  been  developed  to  aggregate  observational  ratings  of  behavioral  parameters 
and  physiologic  measures  into  one  pain  assessment  score  [1, 4,  9-11].  These  tools  are  superior 
to  unimodal  methods  [4]  but  are  subjective  in  nature  and  require  regular  repeat  assessment  by 
care  providers.  Automated  multimodal  pain  assessment  has  great  potential  for  improving  the 
quality  of  pain  management  care.  Recently,  Worley  et  al.  demonstrated  that  a  multimodal 
assessment  tool  could  detect  noxious  heel  lance  and  touch  stimuli  in  infants  with  100% 
sensitivity  and  specificity  [12].  This  assessment  tool  incorporated  videography,  EEG,  EMG,  and 
electrocardiograms  (ECG),  along  with  measures  of  total  hemoglobin  concentration,  respiration 
rate,  and  oxygen  saturation.  This  provided  an  important  demonstration  of  the  efficacy  of 
objective  multimodal  pain  assessment.  However,  the  system  developed  by  Worley  et  al.  was  not 
designed  for  continuous  monitoring  of  clinical  pain.  In  particular,  analysis  was  performed  offline 
and  subjective  analysis  of  videography  was  used  to  assess  behavior. 

Based  on  interviews  with  experts  in  clinical  pain  management,  discussions  with  project 
consultants  from  the  Departments  of  Anesthesiology  and  Surgery  at  the  University  of  Arizona, 
and  emerging  trends  in  the  scientific  literature,  we  identified  two  behavioral  and  four 
physiological  parameters  that  are  correlated  with  acute  pain  (Table  1).  Therefore,  it  is  critical  to 
combine  behavioral  pain  assessment  with  physiological  monitoring,  and  to  interpret  behaviors  in 
a  patient  specific  context  [2].  The  behavioral  and  physiological  variables  that  are  monitored  by 
our  wearable  sensor  system  are  listed  in  Table  1  and  discussed  in  detail  in  subsequent 
sections. 


Table  1:  Pain  indicators  measured  by  MoPASS 


Parameter 

Sensor  Unit 

Sensor  Modality 

Behavioral 

Facial  Expression  (grimace,  clenching) 

Forehead 

Electromyography 

Movement  (restlessness,  guarding) 

Forehead  and  Finger 

Accel  erometry 

Physiological 

Perspiration 

Forehead  and  Finger 

Skin  Conductance 

Heart  Rate 

Finger 

Optical 

Skin  Temperature 

Forehead  and  Finger 

Thermocouple 

Pain  related  electrocortical  activity 

Forehead 

Electroencephalography 

Behavioral  Indicators: 

The  primary  behavioral  categories  used  to  help  identify  pain  are  facial  expression  (such  as 
grimacing,  frowning,  and  wincing)  and  body  movement  (such  as  restlessness  and  guarding)  [1, 
2,  4,  5,  13].  To  quantify  facial  expression  we  use  surface  electromyography  (EMG)  in  the 
forehead  sensor  node  to  record  electrical  potentials  on  the  forehead  and  temples.  Bandpass 
filtering  and  blind-source  separation  techniques  can  be  used  to  isolate  muscle  activity  (e.g., 
from  the  occipitofrontalis,  zygomaticus,  or  temporalis)  and  then  the  activity  time-course  can  be 
used  to  distinguish  normal  muscle  activity  (e.g.,  blinking  or  chewing)  from  facial  tension  (e.g., 
grimace,  wincing,  frowning).  We  have  previously  used  similar  techniques  to  separate  neck 
muscle  activity  from  electrocortical  activity  during  human  locomotion  [14-17],  Restlessness  can 
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be  quantified  with  inertial  sensors  on  the  forehead  and  hand  using  algorithms  that  are  modified 
from  those  we  currently  use  to  quantify  physical  activity  and  gait  in  our  commercially  available 
wearable  inertial  sensor  systems  (LEGSys™  and  PAMSys™).  Our  target  patients  are  non- 
communicative  and  often  non-ambulatory;  nevertheless,  restlessness  (i.e.,  persistent 
movements  and  adjustments  in  the  bed  or  chair)  are  an  important  behavioral  pain  indicator. 

Physiological  Indicators: 

Several  physiological  indicators  have  been  used  to  assess  of  pain.  Increases  in  heart  rate, 
perspiration,  and  skin  temperature  (as  well  as  increased  variability  in  these  parameters)  may  all 
be  indicative  of  pain  [18-21].  MoPASS  measures  heart-rate  using  optical  sensors  in  the  finger 
sensor  unit,  and  perspiration  on  the  finger  and  the  forehead  based  on  skin  conductance. 
Finally,  MoPASS  measures  skin  temperature  on  both  the  forehead  and  finger  using 
thermocouple  based  skin  contact  temperature  sensors. 

Cortical  Representation  of  Pain: 

Advances  in  neuroimaging  are  rapidly 
elucidating  the  cortical  pain  network,  which 
includes  structures  in  the  sensorimotor, 
insular,  anterior  cingulate,  and  prefrontal 
cortex  [22-27]  (Figure  7).  Many  neuroimaging 
techniques  have  evaluated  brain  responses  to 
controlled  acute  pain  (see  [25]  for  a  review). 

The  only  noninvasive,  wearable,  and  fieldable 
method  for  monitoring  brain  responses  to  pain 
is  electroencephalography  (EEG).  EEG 
measures  electrical  potentials  on  the  scalp 
generated  by  synchronous  firing  of  cortical 
neurons.  EEG  signals  that  are  time-locked  to  a 
short  duration  pain  stimulus  (i.e.,  event-related 
potentials)  have  been  widely  studied  [28]. 

While  informative,  these  techniques  are  not 
generally  useful  for  assessment  of  clinical  pain  [25,  28].  A  few  studies  have  demonstrated  that 
changes  in  continuous  EEG  spectral  content  correlate  with  self-reported  pain  scores  [25,  29-31]. 
These  studies  hold  great  promise  for  clinical  pain  assessment.  Most  recently,  Nir  et  al. 
demonstrated  that  increases  in  peak  alpha  frequency  (8-12  Hz)  on  the  bilateral  temporal  scalp 
during  tonic  noxious  temperature  exposure  were  correlated  with  self-reported  pain  score.  When 
combined  with  other  behavioral  and  physiological  indicators  of  pain,  continuous  EEG  monitoring 
may  provide  an  invaluable  tool  for  pain  assessment  in  the  non-communicative  patient. 

MoPASS  measures  electrocortical  activity  using  surface  electrodes  in  the  forehead  sensor  unit. 
Based  on  the  location  of  these  surface  electrodes  we  are  able  to  record  electrocortical  activity 
from  the  medial  and  dorsolateral  prefrontal  cortex,  which  are  critical  regions  in  the  cortical  pain 
network  [22-27].  MoPASS  software  enables  the  collection  and  storage  of  raw  EEG  data  as  well 
as  alpha-band  (8-12  Hz)  and  beta-band  (12-30  Hz)  spectral  power.  Further,  patient  specific 
baseline  values  can  be  recorded  and  stored  in  MoPASS  software  for  later  comparison  to 
spectral  parameters  during  periods  of  acute  pain.  These  fluctuations  from  baseline  will  serve  as 
input  to  our  multimodal  pain  assessment  algorithm  that  will  be  developed  based  on  the  clinical 
trials  conducted  during  Phase  II. 


Figure  7:  Schematic 
the  human  cortical  and 
subcortical  pain 
processing  network. 

From:  Price,  D.P. 
(2000)  Science.  288, 
1769. 
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Second  Phase  I  Outcome:  Development  of  Functional  Prototype  of  MoPASS 

The  majority  of  our  efforts  during  Phase  I  were  related  to  the  development  of  a  functional 
prototype  MoPASS  system.  The  design  objectives  of  this  system,  which  included  hardware  and 
software,  were  to  wirelessly  record  and  display  synchronized  real-time  data  from  the  forehead 
and  finger  sensors,  including  EEG,  skin  temperature,  movement  (i.e.,  acceleration),  heart  rate, 
and  skin  conductance.  The  prototype  system  we  developed  in  Phase  I  is  fully  functional  and 
ready  to  be  used  immediately  in  clinical  studies.  Phase  II  engineering  efforts  will  be  devoted  to 
form-factor  optimization  and  design  for  manufacturing.  During  Phase  II,  off-the-shelf 
components  used  in  Phase  I  will  be  replaced  with  custom  manufactured  hardware.  To  see  video 
of  the  MoPASS  system  being  used  to  record  and  display  data  in  real-time  visit 
www.biosensics.com/mopass/  and  enter  the  password  DHP12-015. 

The  following  Gantt  chart  was  used  during  the  six  months  of  Phase  I  development.  The 
remainder  of  this  section  will  go  into  more  detail  about  each  stage  of  the  development  process. 


Table  2:  Gantt  Chart  of  Phase  I  development  stages 


Months 

1 

2 

3 

4 

5 

6 

Development  of  prototype  sensor  units 

Hardware  design  and  firmware  development 

MoPASS  PC  Software 

Validation  of  prototype  system 

Report  preparation  and  Phase  II  proposal 

Stage  1:  Development  of  Prototype  Sensor  Units 

Task  1:  Identification  of  MoPASS  Sensors 

A  design  matrix  (Table  3)  was  used  to  identify  the  final  sensors  that  would  make  up  the 
MoPASS  integrated  system. 


Table  3:  Design  matrix  used  to  finalize  sensors  used  in  MoPASS  system 


SENSOR  MODALITY  OPTIONS 

i— 

z 

LU 

2 

LU 

> 

O 

BioSensics  PAMSys 

o 

LU 

LU 

NeuroSky 

Emotiv  EEG 
NeuroHeadset 

Interaxon  Muse 

BLOOD  OXYGEN 

SATURATION  & 

PULSE 

NellCor  Pulse 

Bluetooth  Finger 

Pulse  Oximetry 

SpaceLabs  Pulse 

Oximeter 

SKIN 

Shimmer  Research 

Equivital  GSR 

SKIN 

c n 

CD 

<55  £ 

i  i 

LU  J 

o  i= 

Y 

^  Smith  Medical 

Temperature  Probe 

■<  PA  SPORT 

Skin/Surface 

Data  quality 

Y 

Y 

N 

Y 

Y 

Y 

Y 

Y 

N 

Wearability 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

Y 

N 

N 

Cost 

Y 

N 

N 

N 

N 

Y 

N 

Y 

N 

Y 

Y 

Y 

Ease  of 
Integration 

Y 

Y 

N 

N 

Y 

Y 

Y 

Y 

N 

Y 

Y 

Y 

Compactness 

Y 

Y  Y  Y 

Y 

Y 

Y 

Y  Y 

Y 

N 

N 

8 


In  selecting  the  final  sensors,  we  were  most  interested  in  the  quality  of  the  data  and  how  easily 
and  comfortably  the  sensor  could  be  worn.  Other  factors  that  influenced  our  selection  were  low 
cost,  ease  of  integration  with  our  existing  capabilities  and  a  sensor  that  was  relatively  small  in 
size.  A  ‘Y’  denotes  that  the  sensor  met  a  specific  criteria  whereas  an  ‘N’  denotes  that  the  sensor 
failed  to  meet  the  criteria. 


GUI). 


We  elected  to  use  our  own  PAMSys  for  recording  movement.  PAMSys  was 
proven  to  work  well  as  a  product  for  body  motion  analysis  as  a  platform  for 
long-term  physical  activity  monitoring.  In  addition,  PAMSys  is  capable  of 
taking  external  analog  and  digital  signals  from  other  sensors  in  MoPASS 
system  (e.g.  GSR  and  thermistor).  Thus,  choosing  PAMSys  makes 
integration  of  these  sensor  readings  easy  and  reliable.  Lastly,  PAMSys  is 
equipped  with  Zigbee  and  Bluetooth  wireless  capability,  which  makes  it  easy 
to  offload  data  wirelessly  to  nearby  base  station  (i.e.  laptop  running  MoPASS 


We  elected  to  use  Mind  Band  (NeuroSky,  San  Jose  CA)  for  recording 
EEG.  NeuroSky  is  the  global  leader  in  low-cost  dry-electrode  EEG 
technology.  Their  ThinkGear  EEG  module  is  used  in  more  than  1  million 
consumer  EEG  devices.  This  level  of  broad  adaptation  gives  us  high 
confidence  in  their  technology.  Unlike  many  wireless  EEG  providers, 
NeuroSky  offers  a  comprehensive  SDK  and  access  to  both  raw  data  and 
algorithmically  optimized  data.  Specifically,  the  NeuroSky  chipsets 
perform  A/D  conversion,  signal  amplification,  off-head  detection,  EMG 
noise  filtering,  50/60  Hz  line  noise  filtering,  and  pass-band  filtering  (delta, 
theta,  low/high  alpha,  low/high  beta,  and  gamma).  These  pass-band  filtered  signals  will  be  used 
directly  in  our  MoPASS  algorithms.  Specifically,  alpha-  and  beta-band  signals  from  electrode 
locations  above  the  medial  and  dorsolateral  prefrontal  cortex  will  serve  as  input  to  our 
multimodal  pain  assessment  algorithms;  these  are  critical  regions  in  the  cortical  pain  network. 
Finally,  the  patented  NeuroSky  eSense  signal  processing  toolbox  has  great  potential  to  assist  in 
the  development  of  our  MoPASS  algorithms.  Specifically  eSense  has  built  in  eye-blink  detection 
and  can  quantify  level  of  attention.  Eye-blink  detection  will  be  used  in  the  behavioral  indicators 
portion  of  our  multi-modal  algorithm  and  level  of  attention  may  prove  to  be  beneficial  as  a  neuro- 
cognitive  indicator  (clinical  testing  in  Phase  II  will  evaluate  the  efficacy  of  this  approach).  For  all 
of  these  reasons  NeuroSky  was  the  obvious  best  supplier  of  EEG  hardware  for  the  MoPASS 
System.  We  have  already  discussed  this  project  with  development  engineers  at  NeuroSky  and 
have  executed  a  Developer  Agreement. 


We  elected  to  use  a  Bluetooth  Finger  Pulse  Oximeter  for  recording  blood 
oxygen  saturation  and  pulse.  This  pulse  oximeter  was  chosen  because  it  met 
all  five  of  our  selection  criterion.  The  Bluetooth  functionality  makes  it  easy  for 
us  to  integrate  the  sensor  with  our  capabilities  and  control  the  quality  of  the 
data  received.  In  comparison  to  the  other  sensors,  we  preferred  this  sensors’ 
size  and  method  of  attaching  it  to  the  user.  Lastly,  this  was  the  cheapest 
sensor  in  comparison  to  the  alternatives.  For  these  reasons,  this  sensor  will 
be  used  in  the  final  integration  of  the  MoPASS  system. 


We  elected  to  use  Shimmer  GSR  sensor,  a  validated  biomedical-oriented 
research  application,  for  recording  skin  conductance.  Shimmer  GSR 
sensor  interfaces  with  PAMSys  through  an  existing  daughter  circuit  board 
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designed  and  is  manufactured  to  mate  directly  with  PAMSys  motion  sensor  board.  This  reduces 
engineering  time  for  integrating  GSR  sensor  reading  into  MoPASS  integrated  system.  In 
addition,  Shimmer  GSR  sensor  has  a  pre-designed  case  that  encloses  the  daughter  card  and 
the  PAMSys  sensor,  which  protects  the  circuit  boards  and  provides  good  usability  and 
wearability.  Lastly,  the  Shimmer  GSR  sensor  circuit  is  configurable.  For  example,  it  supports 
four  digitally  controlled  measurement  ranges  which  developers  use  to  ensure  accurate 
measurements  and  it  allows  gain  selection  to  be  hard-coded  or  manually  selected. 


We  elected  to  use  a  GE  MA  Series  Thermistor  for  recording  skin 
temperature.  This  thermistor  was  chosen  because  it  met  all  five  of 
our  selection  criterion.  The  main  reason  we  chose  this  thermistor 
over  the  others  is  because  it  is  a  medical  grade  product  meant  to  be 
worn  and  collect  data.  Thus,  our  wearbility  and  quality  of  data 
criterions  were  met.  In  addition,  this  thermistor’s  compact  size  was 
more  attractive  than  the  alternatives.  Lastly,  the  low  cost  and  easy- 
to-integrate  capabilities  ultimately  led  us  to  select  this  sensor  for  the 
final  integration  of  the  MoPASS  system. 


Task  2:  Sensor  Hardware  Development 

The  aforementioned  sensors  were  used  in  the  hardware  development  of  the  wrist  unit  for  the 
MoPASS  system,  as  shown  in  Figure  8. 
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in  conductance) 
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j  Bluetooth  PulseOx 
Finger  Unit 


Figure  8:  MoPASS  Wrist  Unit  for  real-time  measurement  and  streaming  of  movement,  skin 
conductance,  skin  temperature,  heart  rate,  Sp02  readings  over  Bluetooth  wireless  connections  to 
the  monitoring  station. 


The  wrist  unit  is  equipped  with  a  3-axis  accelerometer  for  measuring  hand  movement,  a 
thermistor  for  measuring  skin  temperature,  a  GSR  sensor  for  measuring  skin  conductance,  a 
pulse  oximeter  for  measuring  heart  rate  and  blood  oxygen  saturation  level  (Sp02).  The  main 
data  communication  unit  is  BioSensics’  PAMSys  device,  designed  for  physical  activity 
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monitoring.  In  addition  to  3-axis  accelerometer,  PAMSys  is  equipped  with  2GB  flash  data 
storage,  Zigbee  and  Bluetooth  wireless  data  transceiver,  USB  docking  capability  for  device 
charging  and  fast  data  off-loading/backup.  Thermistor,  GSR,  and  accelerometer  sensor 
readings  are  acquired  through  the  analog-to-digital-converter  (ADC)  ports  on  PAMSys’  main 
processor  (a  MSP430  micro-controller  form  Texas  Instrument)  with  12-bit  resolution.  The 
sampling  rate  is  configurable  through  firmware  settings  and  is  currently  set  to  be  100Hz. 


Stage  2:  Hardware  Design  and  Firmware  Development 

Task  1:  Design  of  MoPASS  System  Architecture 

We  defined  MoPASS  system  to  be  consisted  of  a  MoPASS  Desktop  Application,  the 
Forehead/Head  Unit,  and  the  Wrist  Unit,  as  shown  in  Figure  9.  The  Forehead  Unit  records  3- 
channel  EEG,  skin  temperature,  and  head  movements.  The  Wrist  Unit  measures  heart  rate, 
Sp02,  skin  temperature,  skin  conductance,  and  hand  movements.  Through  Bluetooth,  a 
MoPASS  Desktop  Application  receives,  parses,  and  analyzes  the  collected  samples  from 
Forehead  Unit  and  Finger  Unit.  Using  data  from  multiple  wearable  sensors,  MoPASS  derives  a 
single  score  for  automatic  and  rapid  assessment  of  pain  intensity  and  delivers  the  objective  pain 
assessments  to  clinicians.  We  detail  each  component  of  the  system  in  the  following  sections. 
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Figure  9:  Overall  architecture  of  MoPASS  system.  Forehead  Unit  and  Finger  Unit  both 
contain  multiple  sensors,  sample  them,  and  communicate  with  preliminary  MoPASS 
Desktop  Application  software  in  real-time  over  Bluetooth  wireless  links. 


Task  2:  MoPASS  Sensor  Platform  Development 
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We  developed  wireless  head 
sensor  unit  and  wrist  sensor 
unit  that  are  used  to  measure 
movement,  EEG,  pulse,  blood 
oxygen  saturation  (Sp02),  skin 
temperature,  and  perspiration 
of  subjects.  The  data  collected 
by  these  sensors  will  be  used 
to  extract  various  pain 
parameters  and  to  estimate  the 
objective  pain  level.  The  tasks 
required  for  developing  the  the 
sensor  units  include 

development  of  a  sensor 
firmware  for  data  collection 
and  wireless  data 

communication.  The  sensors 
are  required  to  be  small  and  lightweight,  capable  of  automatic  wireless  data  transmission 
without  any  attention  from  the  user. 

In  order  to  provide  real-time  measurement  of  the  required  pain  assessment  parameters,  the 
forehead  sensor  hardware  includes  EEG  electrodes,  analog  electronic  interfaces,  a  low  power 
processor  for  signal  processing,  and  a  Bluetooth  transceiver.  The  finger  sensor  unit  includes  a 
tri-axial  accelerometer  and  an  external  fingertip  sensor  (connected  through  a  removable 
daughter  board)  to  measure 
pulse  and  Sp02.  For  real-time 
processing  and  transmission  of 
sampled  signals,  we  include  a 
2.4GHz  Bluetooth  transceiver 
with  an  embedded  micro¬ 
controller  in  the  sensor.  As 
shown  in  Figure  10,  the  sensor 
units  include  a  Bluetooth 
transceiver  with  an  embedded 
micro-controller  for  real-time 
data  transmission  and 
processing  of  sampled  signals. 

For  control  and  wireless 
communication  capabilities,  we 
have  selected  Bluetooth  (IEEE 
802.15.1)  to  support  real-time 
data  transmission. 

Figure  1 1  shows  the  firmware 
architecture  that  will  be  used  to  operate  the  wrist  sensor  unit.  We  chose  TinyOS  to  be  the 
operating  system  for  our  firmware.  TinyOS  provides  critical  networking  services  such  as  media 
access  control  (MAC)  protocol  as  well  as  wireless  time  synchronization  (e.g.  FTSP).  The 
MoPASS  sensor  firmware  application  schedules  sampling  timing  and  stores  the  collected 
sample  in  a  local  buffer.  The  radio  scheduler  then  sends  the  data  from  the  buffer  in  the  First-In- 
First-Out  (FIFO)  order.  In  addition,  the  firmware  architecture  includes  a  fault  tolerance  layer  to 
handle  hardware  malfunction  (e.g.  sensor  failure  or  wireless  transmission  failure).  This  layer 
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Figure  11:  Overall  firmware  architecture  of  the  wrist  sensor. 
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Figure  10:  Hardware  architecture  of  the  MoPASS  wrist  unit. 
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reports  unexpected  behaviors  to  MoPASS  Desktop  Application  so  that  users  are  notified  about 
potential  system  failures  and  can  take  actions  to  reboot  or  replace  malfunctioning  sensor  units. 


Task  3:  Streaming  Firmware 

We  developed  streaming  firmware  for  PAMSys  to  collect  real-time  data  from  the  wrist  unit.  The 
PAMSys  has  accelerometer  sensors  sampling  at  40Hz,  which  can  be  configured  to  be  up  to 
1000Hz.  The  firmware  reads  accelerometer  data  through  three  ADC  channels,  and  saves  it  into 
an  internal  buffer  to  be  transferred  to  our  desktop  software. 

We  have  completed  the  development  and  integration  of  sensing  and  wireless  data  streaming 
features  from  PAMSys-based  wrist  unit,  finger-worn  pulse  oximeter,  and  NeuroSky  EEG 
MindBand.  PAMSys  firmware  drives  on  measurements  of  3-axis  acceleration  of  the  subject’s 
hand,  skin  temperature,  and  skin  conductance.  We  have  developed  the  firmware  driver  that 
samples  the  above  sensors  at  up  to  100Hz.  The  sampling  rate  will  be  made  configurable  from 
the  MoPASS  Desktop  Application  Software.  PAMSys  firmware  also  implements  Bluetooth 
communication  with  the  laptop  computer  running  MoPASS  Desktop  Application  Software. 

Bluetooth  Finger  Pulse  Oximeter  implements  a  Bluetooth  communication  protocol  based  on 
Radio  Frequency  Communication  (RFCOMM).  We  were  able  to  read  the  data  stream  from  the 
sensor  from  any  PC/Macs/Tablet  computers  with  a  Bluetooth  transceiver.  The  Bluetooth  Finger 
Pulse  Oximeter  sends  optical  measurements  in  a  7-byte  message  format  at  100Hz.  Also,  the 
Bluetooth  Finger  Pulse  Oximeter  measures  Pulse  and  Sp02  information  and  sends  the  collected 
data  at  115200  bps  to  the  on-board  Bluetooth  transceiver.  In  a  single  message,  the  pulse 
oximeter  sends  the  subject’s  Sp02,  heart  rate,  and  plethysmogram  data  (each  sample  of  which 
is  6  bits),  as  well  as  sensor  connectivity  and  status  (each  of  which  is  1  bit). 

NeuroSky  EEG  MindBand  also  transmits  data  through  Bluetooth.  MindBand  measures  and 
transmits  EEG  alpha,  beta,  gamma,  theta,  and  delta  waves,  as  well  as  connectivity  and  battery 
status.  The  NeuroSky  EEG  MindBand  firmware  samples  data  in  512Hz. 

For  the  wrist  sensor  unit  to  communicate  with  the  MoPASS  Desktop  Application  (MoPASS  GUI), 
we  designed  and  implemented  a  simple  command-response  communication  protocol  for 
MoPASS.  The  protocol  is  implemented  both  in  the  firmware  of  the  wrist  sensor  unit  as  well  as  in 
the  MoPASS  GUI  software.  We  describe  the  protocol  in  detail  below. 

After  powering  up,  the  wrist  sensor  unit  enters  into  idle  mode  and  waits  for  incoming  commands 
from  the  MoPASS  GUI.  The  following  commands  are  supported: 

Status  Inquiry  (STATUSJNQ) 

Current  Sensor  Reading  (SAMP) 

Start  Streaming  (START_STREAM) 

Stop  Streaming  (STOP_STREAM) 

Set  Sampling  Rate  (SET_SAMPRATE) 


Status  Inquiry  (STATUSJNQ)  command  allows  the  MoPASS  GUI  to  receive  the  following 
information  from  the  wrist  sensor  unit:  physical  sensor  status  (i.e.  report  status  of 
accelerometer,  thermistor,  and  skin  conductance  sensor),  battery  level,  sampling  ranges,  and 
sampling  rate.  The  MoPASS  GUI  always  transmits  this  command  when  the  wrist  sensor  is  first 
connected  to  the  PC.  Current  Sensor  Reading  (SAMP)  command  will  trigger  sampling  of  one 
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data  point  from  each  sensor  type  from  the  wrist  sensor  unit  and  transmission  of  the  measured 
values  back  to  the  PC  to  the  MoPASS  GUI  over  Bluetooth  wireless  connection.  The  Start 
Streaming  command  (START_STREAM)  starts  the  continuous  sampling  and  data  streaming 
from  the  wrist  sensor  unit  to  the  PC  at  the  predefined  sampling  rate,  set  to  100Hz  by  default. 
The  Stop  Streaming  command  (STOP_STREAM)  stops  the  data  streaming  and  set  the  wrist 
sensor  unit  back  to  the  idle  mode.  The  Set  Sampling  Rate  command  (SET_SAMPRATE) 
configures  the  wrist  sensor  unit  to  sample  at  one  of  the  following  supported  frequencies:  10Hz, 
50Hz,  100Hz,  125Hz,  200Hz,  and  250Hz.  The  above  commands  allow  the  MoPASS  GUI  to 
dynamically  control  and  query  sensor  status,  start/stop  data  streaming,  and  configure  sampling 
rate. 


14 


Stage  3:  MoPASS  PC  Software 


Task  1:  MoPASS  Desktop  Application  Design  and  Development 


non 


Figure  12:  Tentative  Design  of  MoPASS  Desktop  Application. 


MoPASS  Desktop  Application  will  be  able  to  perform  multi-modal  pain  assessment  based  on  the 
collected  data  from  head  and  wrist  sensor  units.  MoPASS  desktop  application  shows  a  series  of 
real-time  sensor  data,  a  pain  indicator,  as  well  as  panels  to  manage  data  from  multiple  patients. 
Figure  12  illustrate  the  preliminary  version  MoPASS  graphical  user  interface  (MoPASS  GUI). 
MoPASS  GUI  will  provide  features  that  allow  clinicians  to  export  raw  data,  export  analyzed  data, 
save  and  print  the  analyzed  results.  MoPASS  Desktop  Application  is  developed  in  Java 
programming  language  and  will  be  cross-platform.  At  this  stage  we  plan  is  to  support  Microsoft 
Windows  and  MacOS  platforms.  MoPASS  Desktop  Application  is  designed  to  run  on  desktop 
and  laptop  computers.  However,  with  minimal  effort,  we  can  extend  its  usage  to  mobile 
platforms,  such  as  Android  or  Apple  iPad  tablet  computers. 


Task  2:  Mockup  GUI 
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Figure  13:  Development  version  of  the  Desktop  MoPASS  GUI. 


This  screenshot  shows  our  development  version  of  the  Desktop  GUI.  As  we  mentioned  in  the 
previous  section,  our  desktop  software  shows  corresponding  signals:  EEG,  Accelerometer,  Skin 
temperature,  Skin  conductance,  Pulse,  Oxygen  Rate.  The  desktop  program  is  developed  by  the 
following  MVC  (Model-View-Controller)  model  in  JavaFX,  as  JavaFX  is  known  to  be  a  well- 
designed  framework  for  MVC.  Right  under  the  menu  bar,  we  show  sensors  and  their 
connectivity.  Users  can  click  the  button  next  to  them  to  connect  and  disconnect.  Once  all 
sensors  are  connected,  hitting  on  the  ‘Start’  button  initiates  real-time  data  streaming  and 
collection.  Users  can  stop  tests  at  any  time  by  hitting  on  the  ‘Stop  Test’  button.  All  the  collected 
data  can  be  exported  to  a  file  (e.g.,  Comma  Separated  Value  file).  At  the  bottom,  we  show  a 
pain  level  indicator  by  using  a  color  bar:  leftmost  corresponding  to  the  lowest  pain  level  and 
rightmost,  the  highest.  The  ‘Analyze’  button  next  to  the  bar  can  give  users  pain  level  at  any  point 
in  experiments. 


Task  3:  Desktop  Application  Software  Development 

To  connect  and  stream  data  from  the  sensors,  users  first  need  to  find  their  sensors.  Our 
Desktop  software  provides  an  interface  to  find  those  sensors  and  save  needed  information.  The 
following  figures  show  screenshots  of  this  feature. 
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Figure  14:  Screenshot  showing  how  to  find  and  connect  sensors. 


As  Figure  14  shows,  by  clicking  on  the  “Find  Sensors”  button,  users  can  find  their  sensors  and 
by  clicking  on  the  “Save”  button,  they  can  save  the  discovered  sensors’  information  to  a  file. 
Therefore,  users  do  not  need  to  repeat  finding  sensors  afterwards,  unless  they  change  their 
sensors. 

Although  the  listed  devices  use  Bluetooth  for  communication,  some  vendors  do  not  allow 
developers  to  access  directly  through  Bluetooth.  Rather,  they  allow  developers  to  access  data 
through  RS-232  serial  communication  protocol.  The  protocol  was  originally  developed  for 
communicating  with  peripheral  devices  through  serial  cables.  As  the  serial  protocol  is  prevalent 
in  popular  operating  systems,  Bluetooth  drivers  in  OS  emulate  the  serial  communication. 
Therefore,  developers,  after  identifying  which  serial  ports  their  devices  operating  on,  can  read 
and  write  data  through  the  serial  ports.  For  example,  “COMO”  stands  for  a  serial  port  used  to 
communicate  with  EEG  Head  Sensors.  In  summary,  Wrist  Sensors  and  EEG  sensors  work  in 
serial  protocol  whereas  Bluetooth  Finger  Pulse  Oximeter  works  in  direct  Bluetooth 
communication. 

To  connect  Bluetooth  Finger  Pulse  Oximeter,  we  use  Bluecove  as  the  underlying 
communication  library,  implementing  Java  Bluetooth  standard  JS-82  [13].  Originally  developed 
by  Intel  Corporation,  Bluecove  is  now  widely  used  in  Java  Bluetooth  application  and  supports 
multiple  Bluetooth  communication  protocols,  including  RFCOMM  (the  one  that  our  devices’ 
Bluetooth  use). 
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Figure  15:  Real-time  plotting  in  development  version  of  Desktop  GUI 


Moreover,  the  GUI  displays  connectivity  information  by  using  colors  on  a  diagram.  As  shown  in 
the  figure,  only  the  sensors  connected  to  the  GUI  present  its  colors  in  green  (Pulse  Oximeter 
Finger  sensor),  whereas  disconnected  devices  show  their  colors  in  red  (top  of  the  figure).  Also, 
as  presented  in  the  figure,  the  sensors  not  connected  to  our  desktop  program  do  not  visualize 
any  data.  We  finished  visualizing  Pulse  Oximeter  Finger  sensor  and  NeuroSky  EEG  MindBand. 

We  developed  the  MoPASS  GUI  and  have  included  the  communication  module  that  implements 
both  a)  the  protocol  described  in  Task  1  and  b)  the  proprietary  communication  protocol 
supported  by  NeuroSky’s  MindBand.  After  this  module  is  completed,  we  were  then  able  to  query 
the  status  and  to  control  the  sampling  status  of  both  sensor  units  in  real  time  using  the  MoPASS 
GUI.  We  also  implemented  an  extensible  sensor  data  parsing  component  in  the  MoPASS  GUI. 
This  allows  a  flexible  architecture  for  incorporating  or  removing  specific  sensors  in  the  future 
without  changing  the  architecture  of  the  MoPASS  GUI  software.  Specifically,  the  MoPASS  GUI 
can  successfully  start,  receive,  parse,  and  plot,  as  time  series  data,  of  the  following  sensor  data 
in  real  time: 

EEG  (from  MindBand) 

Wrist  Movement  (3D  acceleration) 

Skin  temperature 

Skin  Conductance 
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Pulse 

Sp02 

The  collected  data  can  be  exported  into  Excel  and  MATLAB  readable  format  for  off-line 
analysis.  Thus,  we’ve  successfully  implemented  real-time  data  streaming  and  data  visualization 
for  all  the  components  that  make  up  the  MoPASS  system. 


Stage  4:  Validation  of  Prototype  System 

We  successfully  validated  the  sensor  data  received  from  the  wrist  and  head  sensor  unit  of  the 
MoPASS  prototype.  This  validation  involved  the  collection  and  analysis  of  real-time  multi-modal 
sensor  signals  from  the  Bluetooth  Finger  Pulse  Oximeter,  the  thermistor,  the  GSR  sensor,  the 
MindBand  sensor  and  PAMSys.  We  were  able  to  obtain  correct  pulse,  Sp02,  temperature,  skin 
conductance,  as  well  as  hand  movement  data.  Figure  16  shows  all  the  sensor  components  of 
the  MoPASS  prototype. 


Figure  16:  MoPASS  prototype  system  consisting  of  (A)  the  head  unit  and  (B)  the  wrist  unit. 

More  specifically,  the  head  unit  comprises  of  the  NeuroSky  MindBand  sensor  and  the  wrist 
unit  comprises  of  the  PulseOx  finger  unit,  the  GSR  electrode,  the  thermistor  and  the 
BioSensics  PAMSys+  GSR  sensing  circuit. 

Figure  17  is  a  screenshot  of  the  MoPASS  Desktop  Application  for  real-time  measurement  and 
streaming  from  both  the  wrist  and  head  components  of  the  MoPASS  system.  The  top  graph 
shows  the  real-time  EEG  measurements  from  the  MindBand  during  a  normal  days-worth  of 
activity.  The  remaining  graphs  depict  measurements  from  the  wrist  unit.  The  second  graph  is 
specifically  gathering  real-time  movement  measurements.  The  third  and  fourth  graph  are 
depicting  skin  temperature  and  conductance,  respectively.  Lastly,  the  fourth  graph  is  gathering 
real-time,  accurate  heart  rate  measures.  We  have  successfully  collected  real-time  signals  for 
the  wrist  and  head  components,  validating  the  MoPASS  prototype  system  as  a  whole.  In  Phase 
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II  of  this  project,  we  will  build  on  the  MoPASS  GUI  as  a  foundation  for  running  the  algorithm  for 
multi-modal  pain  detection  and  assessment. 
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Figure  17:  MoPASS  Desktop  Application  for  real-time  measurement  and  streaming  from  both  the 
wrist  and  head  components  of  the  MoPASS  system. 
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Third  Phase  I  Outcome:  Clinical  Study  Design 

The  third  outcome  of  Phase  I  was  the  design  of  clinical  studies  to  be  carried  out  in  Phase  II  and 
the  identification  of  an  appropriate  clinical  site  and  clinical  investigators.  These  studies  are 
described  in  detail  in  the  Phase  II  work  plan.  In  brief,  we  have  identified  an  excellent  clinical 
research  team  at  the  University  of  Arizona  who  will  partner  with  us  in  these  efforts,  and  we  have 
identified  a  research  plan  involving  long-term  monitoring  of  patients  in  the  medicine  ICU.  This  is 
the  ideal  environment  for  the  clinical  studies  for  several  reasons:  1 )  patients  are  often  in  the  ICU 
for  an  extended  period  of  time  allowing  us  the  opportunity  to  acquire  clear  baseline  signals  and 
then  evaluate  changes  in  response  to  painful  procedures  or  medications,  2)  patients  in  the 
medicine  ICU  often  undergo  painful  procedures  (e.g.,  central  line,  chest  tube,  spinal  tap)  and  it 
is  critical  to  monitor  this  pain,  3)  many  of  these  patient  will  be  intubated,  providing  a  portion  of 
the  sample  population  that  is  non-communicative,  and  4)  the  first  civilian  commercialization 
target  for  MoPASS  will  be  as  a  device  intended  for  use  in  the  ICU. 

Phase  II  Technical  Objectives 

There  are  three  objectives  and  associated  deliverables  of  this  Phase  II  project: 

1.  Clinical  Studies 

Objective :  Conduct  a  clinical  study  using  hardware  developed  during  Phase  I  to  collect 
MoPASS  data  and  gold-standard  clinical  assessments  of  pain  in  the  medicine  ICU,  and  use 
this  data  to  develop  an  algorithm  to  combine  indicators  of  pain  into  a  single  pain  score. 

Deliverables :  There  are  three  deliverables  related  to  this  objective:  1)  the  MoPASS  pain 
intensity  algorithm,  2)  a  filed  U.S.  Patent  Application  covering  the  specific  approach  to  multi¬ 
modal  pain  assessment  that  is  discovered  during  these  trials,  and  3)  a  publication  in  a  peer- 
review  scientific  journal  demonstrating  the  statistical  significance  of  the  correlation  between 
the  MoPASS  pain  intensity  score  and  the  gold-standard  clinical  assessments  of  pain. 

2.  Sensor  Improvements 

Objective :  Improve  the  finger  and  forehead  sensors  by  designing  for  manufacturability  and 
improving  the  form-factor  compared  to  the  system  developed  in  Phase  I. 

Deliverables :  A  manufacturing  plan  including  a  complete  bill-of-materials,  vendor  selection, 
cost-estimates,  and  tooling,  as  well  as  a  first  batch  of  20  manufactured  units. 

3.  MoPASS  Computer  for  Clinical  Use 

Objective :  Develop  a  Windows-based  MoPASS  software  platform  that  runs  on  a  ruggedized 
mobile  personal  computer  and  allows  only  the  MoPASS  software  to  run  on  the  device. 

Deliverables :  A  ruggedized  mobile  computer  that  boots-up  directly  into  the  MoPASS 
software,  pairs  automatically  with  MoPASS  sensors  in  range,  continuously  displays  and  logs 
a  patients’  pain  intensity  score,  and  can  be  configured  to  optionally  deliver  an  auditory  alert 
and/or  SMS  format  alert  in  the  event  that  the  pain  score  exceeds  a  clinician  set  threshold. 
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Conclusion 


Based  on  the  significant  work  that  we  have  completed  during  Phase  I  we  are  in  an  excellent 
position  to  meet  these  objectives.  Specifically,  we  are  ready  to  begin  the  clinical  study  at  the 
outset  of  Phase  II  (following  IRB  approval).  This  will  allow  us  to  pursue  Phase  II  engineering 
developments  in  parallel  with  Phase  II  clinical  studies.  By  the  completion  of  Phase  II  we  will 
have  a  finalized  device  that  is  ready  to  be  used  in  a  comprehensive  clinical  trial  during  Phase  III 
in  anticipation  of  an  eventual  FDA  submission  for  approval  as  a  Class  II  Medical  Device. 
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